{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial 4. Immediate mode\n",
    "\n",
    "In this tutorial we will talk about a cute feature about Caffe2: immediate mode.\n",
    "\n",
    "From the previous tutorials you have seen that Caffe2 *declares* a network, and during this declaration phase, nothing gets actually executed - it's like writing the source of a program, and \"compilation/execution\" only happens later.\n",
    "\n",
    "This sometimes gets a bit tricky if we are in a researchy mind, and want to inspect typical intermediate outputs as we go. This is when the immediate mode come to help. At a high level, what the immediate mode does is to run the corresponding operators as you write them. The results live under a special workspace that can then be accessed via `FetchImmediate()` and `FeedImmediate()` runs.\n",
    "\n",
    "Let's show some examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "from caffe2.python import cnn, core, visualize, workspace, model_helper, brew\n",
    "import numpy as np\n",
    "import os\n",
    "core.GlobalInit(['caffe2', '--caffe2_log_level=-1'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Now, as we have known before, in the normal mode, when you create an operator, we are *declaring* it only, and nothing gets actually executed. Let's re-confirm that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "workspace.ResetWorkspace()\n",
    "# declaration\n",
    "op = core.CreateOperator(\"GaussianFill\", [], \"X\", shape=[3, 5])\n",
    "print('Before execution, workspace contains X: {}'\n",
    "      .format(workspace.HasBlob(\"X\")))\n",
    "# execution\n",
    "workspace.RunOperatorOnce(op)\n",
    "print('After execution, workspace contains X: {}'\n",
    "      .format(workspace.HasBlob(\"X\")))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Entering and exiting immediate mode.\n",
    "\n",
    "Entering immediate mode is easy: you basically invoke `workspace.StartImmediate()`. Since immediate mode has quite a lot of side effects, it would be good to read through the warning message to make sure you understand the implications.\n",
    "\n",
    "(If you don't want to see the messages, pass `i_know=True` to `StartImmediate` to suppress that.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "workspace.StartImmediate()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you have enabled immediate mode, any operators you run will simultaneously be executed in a separate immediate workspace. Note - the main workspace that you are working on is not affected. We designed the immediate workspace to be separate from the main workspace, so that nothing in the main workspace gets polluted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# declaration, and since we are in immediate mode, run it in the immediate workspace.\n",
    "op = core.CreateOperator(\"GaussianFill\", [], \"X\", shape=[3, 5])\n",
    "print('Before execution, does workspace contain X? {}'\n",
    "      .format(workspace.HasBlob(\"X\")))\n",
    "print('But we can access it using the Immediate related functions.'\n",
    "      'Here is a list of immediate blobs:')\n",
    "print(workspace.ImmediateBlobs())\n",
    "print('The content is like this:')\n",
    "print(workspace.FetchImmediate('X'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# After the immediate execution, you can invoke StopImmediate() to clean up.\n",
    "workspace.StopImmediate()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Manually feeding blobs\n",
    "\n",
    "But wait, you say - what if I want to create an operator that uses an input that is \"declared\" but not present yet? Since the immediate workspace does not have the input, we will encounter an exception:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "workspace.StartImmediate(i_know=True)\n",
    "op = core.CreateOperator(\"Relu\", \"X\", \"Y\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "This is because immediate mode, being completely imperative, requires any input to be used to already exist in the immediate workspace. To make the immediate mode aware of such external inputs, we can manually feed blobs to the immediate workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X = np.random.randn(2, 3).astype(np.float32)\n",
    "workspace.FeedImmediate(\"X\", X)\n",
    "# Now, we can safely run CreateOperator since immediate mode knows what X looks like\n",
    "op = core.CreateOperator(\"Relu\", \"X\", \"Y\")\n",
    "print(\"Example input is:\\n{}\".format(workspace.FetchImmediate(\"X\")))\n",
    "print(\"Example output is:\\n{}\".format(workspace.FetchImmediate(\"Y\")))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "workspace.StopImmediate()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## When is immediate mode useful?\n",
    "\n",
    "You might want to use immediate mode when you are not very sure about the shape of the intermediate results, such as in a CNN where there are multiple convolution and pooling layers. Let's say that you are creating an MNIST convnet model but don't want to calculate the number of dimensions for the final FC layer. Here is what you might want to do."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model = model_helper.ModelHelper(name=\"mnist\")\n",
    "# Start the immediate mode.\n",
    "workspace.StartImmediate(i_know=True)\n",
    "\n",
    "data_folder = os.path.join(os.path.expanduser('~'), 'caffe2_notebooks', 'tutorial_data')\n",
    "data_uint8, label = model.TensorProtosDBInput(\n",
    "    [], [\"data_uint8\", \"label\"], batch_size=64,\n",
    "    db=os.path.join(data_folder, 'mnist/mnist-train-nchw-leveldb'),\n",
    "    db_type='leveldb')\n",
    "data = model.net.Cast(data_uint8, \"data\", to=core.DataType.FLOAT)\n",
    "data = model.net.Scale(data, data, scale=float(1./256))\n",
    "data = model.net.StopGradient(data, data)\n",
    "conv1 = brew.conv(model, data, 'conv1', 1, 20, 5)\n",
    "pool1 = brew.max_pool(model, conv1, 'pool1', kernel=2, stride=2)\n",
    "conv2 = brew.conv(model, pool1, 'conv2', 20, 50, 5)\n",
    "pool2 = brew.max_pool(model, conv2, 'pool2', kernel=2, stride=2)\n",
    "\n",
    "# What is the shape of pool2 again...?\n",
    "feature_dimensions = workspace.FetchImmediate(\"pool2\").shape[1:]\n",
    "print(\"Feature dimensions before FC layer: {}\".format(feature_dimensions))\n",
    "\n",
    "fc3 = brew.fc(model, pool2, 'fc3', int(np.prod(feature_dimensions)), 500)\n",
    "fc3 = brew.relu(model, fc3, fc3)\n",
    "pred = brew.fc(model, fc3, 'pred', 500, 10)\n",
    "softmax = brew.softmax(model, pred, 'softmax')\n",
    "\n",
    "# Let's see if the dimensions are all correct:\n",
    "for blob in [\"data\", \"conv1\", \"pool1\", \"conv2\", \"pool2\", \"fc3\", \"pred\"]:\n",
    "    print(\"Blob {} has shape: {}\".format(\n",
    "          blob, workspace.FetchImmediate(blob).shape))\n",
    "# Let's also visualize a sample input.\n",
    "print(\"Sample input:\")\n",
    "visualize.NCHW.ShowMultiple(workspace.FetchImmediate(\"data\"))\n",
    "workspace.StopImmediate()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember, immediate mode is only intended to be used in debugging mode, and are only intended for you to verify things interactively. For example, in the use case above, what you want to do eventually is to remove the feature_dimensions argument and replace it with code that do not depend on immediate mode, such as hard-coding it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Departing words\n",
    "\n",
    "Immediate mode could be a useful tool for quick iterations. But it could also easily go wrong. Make sure that you understand its purpose, and never abuse it in real product environments. The philosophy of Caffe2 is to make things very flexible and this is one example of it, but it also makes you easy to shoot yourself in the foot. Take care :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
